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ABSTRACT 



A cognitive study examined expertise in managerial problem 
solving. Research questions asked what types of managerial knowledge do 
experts and novices use in representing and solving problems, and how do 
knowledge types develop over time? The 115 participants were divided into 9 
levels of expertise: first through fourth year students in the management 
sciences program of Maastricht University and 2-, 5-, 12. 5-, and 25-year 
working experts. Materials consisted of two case descriptions concerning 
real-life business situations. Students had a limited reading time after 
which the text was removed. Subjects carried out the following three 
assignments with the case information: recall of the case studied; a written 
analysis about the situation; and a management diagnosis about the situation. 
Results of the recall exercise suggested clear differences among novices, 
intermediates, and experts in the type of recall. An absolute maximum was 
reached at intermediate level. Experts recalled relatively more relevant 
information. Novices produced more declarative knowledge types than experts 
in the recall, and case analysis exercise, while experts reproduced more 
inferences. (Contains 20 references.) (YLB) 
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Introduction 



Next to the established professions as medicine, law and engineering, the field of 
management sciences is considered as one of the new professions. For a long time 
management sciences were dominated by the view that managers are rational 
technicians or management engineers (Wagner, 1991). This resulted in the 
development of rational, behaviorist approaches to managerial problem solving with 
an emphasis on general principles of problem solving. Content knowledge was 
considered as less important. However, in the 1970s and the 1980s a growing 
skepticism rose about the power of general principles of problem solving (McCall & 
Kaplan, 1985). For example, Mintzberg (1973) found on the basis of observations 
and interviews that managers rarely employ rational approaches. As Mintzberg 
lacked modern tools from cognitive psychology for further examination of managerial 
problem solving, he considered his own study as “sketchy”. More recent, so called 
2 nd generation expertise studies (Schragen, 1994) in the 1980s showed the importance 
of knowledge. In problem solving, experienced managers asked for less information 
and made more inferences from data, than students (E.g., Isenberg, 1986). The 
Isenberg study was one of the first that questioned the importance of general problem 
solving methods in the management sciences. The study was also one of the few in 
this field that applied methods for expertise research similar to those in medicine or 
physics. Basically, studies on managerial problem solving from a cognitive science 
perspective are still limited to a few (See e.g. VanFossen & Miller, 1994; Gijselaers & 
Woltjer, 1997), though a large area of studies on expertise development is available 
within the established professions. Nowadays, the general agreement about 
development of expertise is that the ability to solve problems in a certain knowledge 
domain is not only a result of better heuristics but also depends on one’s knowledge 
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of a specific domain (e.g. Glaser & Chi, 1988; Ericsson & Smith, 1991; Schmidt & 
Boshuizen, 1993). Few studies, however, exist that examine this knowledge of novice 
and expert problem solvers in management sciences. Especially studies are lacking 
that include a large range and number of subject-groups. However, the growing 
importance of research on managerial decision-making and managerial expertise 
requires that studies are conducted that apply frameworks of expertise development as 
derived in mainstream cognitive psychology (Patel & Groen, 1991). 

In many areas where expertise has been studied, consistent patterns have been 
found (Glaser & Chi, 1988) on the development of expertise over time. One of the 
most consistent findings in novice-expert comparisons in the area of medicine is the 
so-called intermediate effect (Schmidt, Boshuizen & Hobus, 1988). Intermediate 
students not only recall more propositions than novices but typically they also recall 
more than experts. The intermediate effect can be characterized as a time stage in the 
development of expertise. The development of expertise in general has a stagelike 
and discontinuous nature, according to findings of Schmidt and Boshuizen (1993) in 
the medical domain. 

The goal of this cognitive study is to examine expertise in managerial problem 
solving. More specific, a (first) research question is: what kind or types of managerial 
knowledge (facts, concepts, principles, diagnoses) do experts and novices use in a) 
representing and b) solving problems in the management sciences? To answer this 
question seven types of knowledge were investigated in recall, analysis and solving of 
problems. A second research question was: how do the knowledge types develop over 
time? To answer this question, nine subject expertise groups were tested (from 
younger novices to older experts). Answers to these research questions will be an 
important input for current management education. If we know how a certain 
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expertise develops over time, education and training can better be adapted to this 



process of knowledge development. 



Method 



Subjects. The present study had 1 1 5 participants, divided into nine different levels of 
expertise: five student groups and four working expert groups (see table 1). 

Table 7: Distribution ofSubjects 



Novice 


Students 


Students 


Students 


Students 


Experts 


Experts 


Experts 


Experts 


Students 


1 th -year 


2 nd -year 


3 th year 


4 lh -year 


2 year 


5 year 


12.5year 


25 year 


18 


14 


22 


16 


16 


8 


6 


8 


7 



All the student groups were all selected out of the management sciences program of 
Maastricht University. Intermediates were called the third and the fourth year 
students (a few months before graduation). The ‘emerging experts’ (two groups with 
on the average two and five years of working experience) were graduates of the 
management sciences program. Finally the last two experts groups were selected on 
the criteria of about 12.5 and 25 year of working experience in the domain of 
management. 



Materials. The materials consisted of two case descriptions, concerning real-life 
business situations. See appendix A for an English version of case A. To enhance the 
reliability and generalizability of the experiment, two cases were used. For each case 
two blank response sheets were included. Both cases were considered to be at 
intermediate level of difficulty, that is end of 3 rd -year students should be able to solve 
them. Each case described managerial issues within an organization. Case A (‘Flex 
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Ltd’) concerned Organizational Development. The issue of Case B (‘Crea Ltd’) was 
about Human Resources Management. To improve external validity, the case content 
was verified by two business consultants. The content of the cases was selected 
within the existing management curriculum. The case content and canonical model 
explanation model (‘solution’) were developed with teachers of the curriculum 
concerned. 

A limited reading time for the cases was set, based on the length of the case. In a 
pilot test it was found that subjects needed on the average 0.4 second per word to 
study a text. The fixed reading time of the cases was calculated as follows. Case A 
(‘Flex Ltd’) contained in the original Dutch text 339 words; 0.4 second a word leads 
to 2.15 minutes. (Case B: 425 words, 2.50 minutes reading time). 

Procedure. Subjects were instructed to study a short case description in a fixed 
period of time, after which the text would be removed. The subjects were told to 
carry out three assignments with the case information shown. The subjects was told 
be in the position of business consultant. This was told to activate prior knowledge, 
in the knowledge domain to be investigated. Subjects were not allowed to consult the 
case description while working on the three assignments that the cases consisted of. 
The first assignment was a recall of the case studied. Subjects were asked to write 
down as much as possible they could remember of the case. As the subjects were not 
aware that they would be asked to recall the case, this is referred to as the ‘ incidental 
recall method’. When the second case was offered, the subjects now were aware to 
expect a recall exercise, hence this second condition could be considered as an 
‘intentional recall method’. To neutralize the recall effect, the two cases (A, B) were 
offered in a mixed order. Next to the recall exercise, the subjects had to provide a 

written analysis about the situation presented in the management case. Finally, a 
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management diagnosis about the situation was required. All subjects received a small 
compensation for their participation. 

Analysis. The original case texts were segmented into propositions (a small, 
meaningful unit of information containing a topic and a connected relation). In 
general, adjectives in sentences form the basis for separate propositions: I.e. the 
sentence ‘Decision-making was getting more complex and tiresome’ contains two 
propositions. The scores for every subject were based on a standard technique of 
proposition analysis (Patel & Groen, 1986). A recalled sentence could contain more 
than one proposition. For every complete proposition, the score ‘2’ was given. 
However, sometimes the subjects recalled only a (substantial) part of a proposition, 
like: ‘Decision-making was getting more complex’. In that case a score 1 was 
provided. 

Both the case recall and case analysis protocols, as produced by the subjects, were 
scored against the original case text as following: 1. Case recall protocols were scored 
on a) Number of literal reproductions of the text (divided into relevant and irrelevant 
propositions by organizational experts) and b) Number of inferences. Inferences were 
defined as knowledge transformations, made on original text-propositions. Examples 
of inferences are conclusions and summaries. 2. Case analysis protocols were scored 
by counting the correct numbers of a) facts, b) inferences, c) managerial concepts, and 
d) diagnosis. A managerial concept was scored when subjects showed to use 
concepts like: “this is a bureaucratic organization”. 
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Results 

For comparing the case instruments, an analysis of variance was carried out for the 
combined results of case A and B. For comparing the two cases, the scores on the 
cases were standardized by using the percentage of propositions recalled. For 
example the scores of the subjects for the variable Total recall was divided by the 
maximum recall score for case A (107), leading to scores in percentages. The total 
recall in percentages of case A and B is depicted in figure 1 : 



Means of Total Recall (Percentages) 




EXPERT LEVEL 



Figure 1: Total recall in percentages of case A and B compared 



To compare the results of the two cases, a univariate analysis was carried out with the 
Factor Level of expertise and the independent variable Total recall score on case A 
and B (see table 2.). No case effect was found (F(l , 1 85) = 0.38, MS e =l,93, p = .846), 
nor an interaction effect between cases A,B and level of expertise (F(8,185) = 0.443, 
MS e =22,65, p = .849). As expected, a significant effect for Level of Expertise 
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(F(8, 1 85) = 13.54, MS e =692,16, p = .000) was found for the total recall score of case 
A and B. These results suggest that the differences in relative scores of the subjects 
on the two cases are not different. Hence, for further analysis, the scores on case A 
and B were combined into one case score. 

Table 2: The results on the two measurement instruments (cases) compared for the 
varable total recall on the two cases. 



Tests of Between-Subjects Effects Dependent Variable: % Total Recall 



Source 


Type III Sum of 
Squares 


df 


Mean Square 


F 


Sig. 


Corrected Model 


5851,080 


17 


344,181 


6,734 


,000 


Intercept 


84569,749 


1 


84569,749 


1654,652 


,000 


CASE Variable 


1,932 


1 


1,932 


,038 


,846 


EXPERTLEVEL 


5537,290 


8 


692,161 


13,543 


,000 


CASE Variable * 


181,205 


8 


22,651 


,443 


,894 


EXPERTLEVEL 












Error 


9455,402 


185 


51,110 






Total 


125073,518 


203 








Corrected Total 


15306,483 


202 









a R Squared = ,382 (Adjusted R Squared = ,325) 



Analysis of the literally recall. For the analyses of the cases, three types of recall 
were distinguished: a) Recall that was relevant related to the problems in the case, b) 
Recall that was not relevant, and c) the Total recall (a+b). Significant effects of level 
of expertise were found for: 

Total recall: F(8,103) = 10,40; MS e = 440,15, p = .000; 

Relevant recall: F(8, 103) = 10,42; MS e = 228,40, p = .000; 

Irrelevant recall: F(8, 103) = 5,53; MS e = 49,64, p= .000. 

The relation between the three average types of recall and the nine levels of expertise 
is depicted in figure 2, showing that after an initial increase, the amount of recall 
actually decreases, beyond the level of intermediate subjects. In other words: the 
shapes of the curves seem to have an ‘inversed U-relation’. 
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Types of recall by Level of expertise 




EXPLEVEL 



Figure 2: Means of produced types of literally recall as a function of expertise 
(combined results case A + B). 



The number of inferences. Results show that the number of inferences continuously 
increases with level of expertise, see figure 3. For the number of inferences a 
significant effect of level of expertise was found (F(8, 1 03) = 27,55, MS e =l 13,15; p = 
. 000 ). 
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Inferences by Level of expertise 




EXPLEVEL 

Figure 3: Inferences by level of expertise (recall exercise). 



Analysis of the problem solving assignment. 

Facts. The number of facts used in problem solving shows an ‘inverted U-relation’. A 
significant effect was found (F(8,105) = 3,089; MS e =12,22, p = .004). 

Inferences. For the number of inferences a significant effect of level of expertise was 
found (F(8,l 05) = 28,31, MS e =69.92, p = .000). As in the recall assignment the number of 
inferences increases continuously with level of expertise. 

Economical concepts. A statistically significant effect of level of expertise was found for 
the use of the number of managerial concepts (F(8,105) = 16,07; MS e =13,86, p = .000). 
Diagnostic accuracy. For the number of correct diagnoses a significant effect of level of 
expertise was found (F(8,105) = 16.02, MS e =10,05, p = .000). 
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Knowledge types by level of expertise 




EXPLEVEL 

Figure 4: Means of counted facts, inferences, concepts and diagnoses as a function of 
expertise level. 

In figure 4 the relation between four different types of knowledge used during 
managerial problem solving is depicted for the nine levels of expertise. Typically the 
relation between level of expertise and a) facts, b) concepts and c) diagnoses shows 
three times an inverted U-form. This implies that after an initial increase, the number 
of facts, concepts and diagnoses actually decreases, beyond the level of intermediate 
subjects. 



Discussion and implications 

The results out of the recall excercise suggest clear differences between novices, 
intermediates and experts in the type of recall. Concerning the amount of totall recall, 
an absolute maximum was reached at intermediate level. The shapes of all the recall- 
curves seem to show an ‘inversed U-relation’. In contrast to novices and students, 
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experts recall relatively more relevant information but very few irrelevant 
information. In the medical domain, Coughlin & Patel (1986) emphasized the higher 
selective perception ability of experts, in contrast to novices who identically represent 
relevant and irrelevant information. Apparently, good problem solvers (experts) 
make a better (more relevant) representation of the problem, as often is found in 
expertise research (Coughlin & Patel, 1986). 

Another general conclusion is that novices produce more declarative knowledge types 
literal (propositions and facts) than experts, both in the recall and the case analysis 
exercise, while the experts groups reproduced more inferences (summaries, 
conclusions). In other words, the findings suggest a shift in use of declarative 
knowledge types by novices toward a more abstract (higher-order) level of expertise 
by experts. This shift, together with the decrease of applied knowledge at the highest 
expert levels can probably be explained by a) compilated or b) encapsulated 
knowledge bases of experts (Schmidt & Boshuizen, 1993). 

1. Knowledge encapsulation. Schmidt & Boshuizen (1993) define knowledge 
encapsulation of experts’ knowledge as the subsumption of lower-level, detailed 
concepts under a smaller number of high-level concepts with the same explanatory 
power. This explanation refers to the idea that experts reason with more condensed or 
packed types of knowledge. Encapsulation is a cognitive process operating on 
declarative knowledge, as it concerns the hierarchical structure of concepts in the 
human mind (Schmidt & Boshuizen, 1993). 

2. The decrease of applied knowledge at the highest expert levels can also be 
explained a result of compiled knowledge bases of experts (Schmidt & Boshuizen, 
1993). Knowledge compilation, in contrast to encapsulation, operates upon 
procedural knowledge. This process is described by Anderson (1987) and explains 
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the phenomenon of packed reasoning. Experts may skip several (automated) phases 
in explaining problems and arguing solutions in case protocols, which can explain 
why experts use less knowledge in case protocols. 

3. Another possible explanation for the decrease of applied knowledge at the experts 
levels may be that experts need less knowledge to solve problems. It is known that 
experts work more efficient (more quickly and directly) than novices because of the 
nature of their well-organized knowledge base (Ericson & Smith, 1991). Intermediate 
students for example often carry out many irrelevant searches in seeking for a right 
solution to solve problems (e.g. Gijselaers & Woltjer, 1997). It is also generally 
known that experts have more experience in recognizing or ‘typical patterns or case 
situations’. These two facts may lead to the idea that experts are better able to use the 
right and necessary knowledge in the right situations, implying that experts use 
knowledge more effective. Probably, this can also explain why experts recall very few 
irrelevant case information. 

A final point to be discussed here is the fact that results suggest gradually and 
continuous shaped curves, in contrast to several other novice-expert studies that used 
three or just two (Van de Wiel, 1997) subject-groups, leading to discontinuous 
graphical shapes of expertise development. The question here is how the shape of 
these graphical curves would change by using a very large number of subject-groups. 
The present study used nine subject-groups. Also in this study the highest experts 
levels had consisted of very experienced managerial experts (on the average more 
than 25 years of working experience), which is much more than usually in expertise 
studies. The results of this study can therefore be considered as a refinement of 
earlier expertise studies. 
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Recommendations for further research. In this research a cross-sectional design was 
used: all nine subject-groups were tested in the same year, due to time limitations. A 
suggestion for further research is to follow the developments of one group, hence 
using a longitudinal design. Further, in this research absolute differences are explored 
between subject-groups that were selected on the criteria of difference in study 
progress or work experience. It would be interesting however to further explore the 
relative expertise, that is exploring differences in performances between individuals 
in one group (See also VanFossen & Miller, 1994) on behave of demographical 
variables such as age and gender or student characteristics (see also Gijselaers & Arts, 
2000 ). 

Implications for education. This study demonstrates for educational practice the 
importance of a focus on the acquisition of content knowledge (besides the acquisition 
of heuristics) in order to become expert in a specific domain. Students should 
however not only be provided with a great amount of domain specific content 
knowledge. The results above show that (especially) intermediate students possess of 
a great amount of declarative knowledge but, compared to experts, seem to be less 
able to apply this knowledge, as they make relatively less operations (inferences) with 
this knowledge. This finding is in agreement with the often heard criticism that 
managerial education delivers graduated students that posses a lot of knowledge but 
are not yet able to use the appropriate knowledge in a business context. Also Van de 
Wiel (1997) concluded that expert knowledge, in contrast of student knowledge, 
seems to be organized in such a way that it is readily applicable in practical settings. 
To overcome this transfer problem a focus not only should be on acquiring 
knowledge, but students should also be trained in using knowledge effectively, such 
as learning to separate relevant from irrelevant knowledge in a given context. 
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Another aspect here is what Leinhardt, McCarthy Young, & Merriman (1995) state in 
the context of transfer. Applying knowledge in education involves labeling, 
differentiating and justifying it. These are typically cognitive activities where 
education is focused on while practical contexts require executing, applying and 
prioritizing knowledge. This may be an explanation why intermediate students 
produced fewer inferences than experts. Hence, the use and integration of declarative 
knowledge into practical situations should be fostered. This can be realized by 
applying knowledge in real-life situations through realistic cases (Christensen, 1987). 
The adoption of the case method in education is one response by business schools to 
the discrepancy between problem solving as it occurs in the context of real 
organizations and as it is described in textbooks. The case method here emphasizes 
understanding the situational context of a business problem (Christensen, 1987), 
which can lead to a better transfer of knowledge. 

Conclusions 

The present study revealed differences in the representation and usage of content 
knowledge between experts and novices, concerning: 

1. The number of knowledge types produced. The results show that experts in 
management sciences in general produced less knowledge types than novices. 
Experts especially produced less: irrelevant recall, literal propositions, concepts and 
diagnoses. The results visually seem to show an ‘inverted U-relation’: after an initial 
increase, the amount of this knowledge decreases, beyond the level of intermediate 
subjects. However, both in the recall and the problem solving assignment the number 
of inferences continuously increased with level of expertise, showing visually a linear 
relation, as the number of inferences produced was higher for every level of expertise. 
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Concerning the amount of knowledge produced an intermediate effect of the 3 rd - and 
4 th -year students appeared for both cases and all knowledge types, except inferences. 

2. Qualitative aspects of the knowledge produced by the subjects in the case 
assignments. The findings suggest a shift in use of declarative knowledge types 
(literal propositions and facts) by novices through an extensive use of all types of 
knowledge by intermediates toward a more abstract (higher-order) level of expertise 
(by using more inferences) by the experts. 

3. Further, the results in general imply that the findings of Schmidt and Boshuizen 
(1993) in the medical domain and the model of expertise development as formulated 
by Chi, Glaser and Rees (1982) can be applied to the management sciences. A 
refinement that can be made here is that the present study suggests expertise 
development curves that are gradually and continuously shaped. This leads to the 
conclusion that the development of expertise is a gradual and continuous process 
(Van de Wiel, 1997). 
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Appendix A: 



The first case used in the experiment, Case A: Flex, Ltd. 

Mr. Fox is director and founder of an employment agency (Flex, Ltd.). More than 25 
local offices are divided in several regions. Most offices are located in the South of 
the Netherlands. The state of Zeeland covers more offices than the state of Limburg. 
Flex showed a remarkable growth: from 70 to 150 employees, sales growing from 
NLG 15 million to NLG 50 million. Average sales of competitors are about $ 60 
million. 

Mr. Fox’s desk contains a growing number of files. Decision-making is getting more 
complex and tiresome. A growing number of files remain closed. Mr. Fox prefers an 
informal organization, employees can just drop in his office. Local directors contact 
Mr. Fox frequently for detailed issues, meanwhile complaining about a lack of central 
policy making and clear company vision. 

Due to the development of new kinds of jobs, Flex decided to specialize. Different 
offices specialize in different job areas. Some offices are specialized in 
administration, industry, general management and health care. This is unlike in the 
past when offices covered all job areas. It was not unusual that even physicians and 
architects acquired new jobs through the offices of Flex company. 

Local offices are not used to this change. Directors of these offices complain about 
their job because it is no longer challenging now that they are only targeting at a 
single profession or job area. It shows that local offices are often getting in conflict 
with other local offices when negotiating with the same company or client. In addition 
some local offices initiated their own market research. Clients (companies looking for 
new personnel) complain that they have to contact different offices for different job 
specialties. 

Mr. Fox considers possibilities to improve the performance of Flex. He thinks about 
splitting shares. The accounting department, however, is convinced that Mr. Fox 
should lower his expenses for public-relations and sponsoring (Mr. Fox considers 
becoming a sponsor for a tennis-club). The accounting department argues that more 
money should be allocated to hiring new people for the local offices, or development 
of a new house-style for the entire organization. The IT-department is convinced that 
the development of a new financial accounting system provides better and faster 
information about the company’s cash-flow. In addition the write-off period for 
computers should be reduced to four years. 
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